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PRINCIPAL  INVESTIGATOR:  Edwins  L.  Rissland, 

TITLE:  Case-Based  Reasoning  in  Mixed  Paradigm  Settings  and  with  Learning 

OBJECTIVE:  In  this  project  we  investigated:  (1)  CBR  in  mixed  paradigm  settings,  in  particular  in  a 
ldaddx>ard-t>ased  tystem,  called  FRAl^  that  generated  various  types  of  explanations  and  arguments 
where  suppcnling  tasks,  such  as  case-  and  rule-tyised  reasoning,  were  dyxuunically  configured  to  reflect 
the  user's  intend^  purposes  for  the  report;  (2)  *pure"  CBR,  particularly  issues  coiKxnung  the  use  of 
multiple  iikUces  and  t)^;>es  of  case  re{sesentations,  in  a  systnn  called  BankXX,  that  used  classic 
heurikic  best-first  search  to  retrieve  iidbmnation  needed  for  case-based  argument;  arul  (3)  the 
application  of  nnachine  learning  techniques  to  core  issues  in  CBR,  such  as  the  problems  of  learning 
indices  and  prototype  cases  and  estinnating  concept  and  theory  drift. 

APPROACH:  We  designed,  built,  and  tested  several  mixed  paradigm,  pure,  and  learning-enhanced 
CBR  tystems  in  dtxnains  where  reasoning  from  past  (precedent)  cases  is  a  or  necessary  tool.  Target 

applicaticm  donudns  itKhided  legal  reasorung,  decision  suj^rt  in  medicine,  and  v  issic  classification 
domains  from  the  UC/Irvir\e  datasets. 

DETAILED  SUMMARY:  Here  we  provide  a  brief  summary  of  the  results  of  our  research.  Fuller 
exposition  can  be  found  in  the  published  research  articles  that  are  iiKluded  in  the  appendix. 

(1)  CBR  in  a  Mixed  Paradigm  Setting:  We  explored  issues  concerning  the  use  of  C3R  in  a  mixed 
paradigm  setting  in  several  blackboard-bas^  systenrs:  GBB-CABARET,  ABISS,  and  particularly, 

FRANK .  Our  goal  was  to  explore  control  issues  and  the  use  of  CBR  in  a  mixed  paradigm  context, 
particularly  a  blackboard-based  architecture.  In  FRANK,  we  focused  on  the  influerKe  of  high  level 
reasoning  goals — such  as  advocacy  or  advice-giving— on  the  details  of  CBR  and  vice  versa,  the  affect  of 
CBR  results  on  higher  level  tasks. 

GBB-CABARET  and  ABISS 

During  fiie  first  year  of  this  project,  we  built  two  precursors  to  FRANK.  Both  were  built  in  the 
Blackboard  Development  (GBB)  environment.  GBB-CABARET  was  a  variant  of  our  previous  CABARET 
system  [Risslatkl  &  Scalak,  1991].  (Work  on  CABARET  was  supported  under  previous  grants.)  It  used 
GBB  to  implement  CABARET'S  agenda-driven  control  strategy.  We  also  ex{dored  various  CBR  and 
related  control  issues  in  ABISS  (A  Blackbc»rd  Integer  Sequence  Solver)  [Ri^Iand  et  al.  1991].  We  built 
upon  our  preliminaiy  experience  with  foese  two  blackbo^-based  tystems  in  bvdiding  FRANK 
[Rissland  et  al.,  1993]. 


The  FRANK  Project 


FRANK  (Flexible  Report  and  ANalysis  System)  demonstrates  dynamic  context-sensitivity  and  use  of 
internal  feedback  in  performing  key  tasks  such  as  retrieval.  FRANK  is  implemented  in  the  Generic 
Blackboard  Development  (GBB)  environment  and  instantiated  in  the  application  domain  of  medical 
diagnosis  and  treatment  of  back  complaints  and  injuries.  FRANK  uses  toth  HYPO-style  and  nearest 
neighbor  CBR  for  case-based  reasoning;  it  uses  OP^  for  rule-based  reasoning. 


As  well  as  using  a  blackboard-based  mixed  paradigm  architecture,  FRANK  is  novel  in  how  it 
dynamically  config  .es  supporting  tadcs — such  as,  case-based  and  rule-based  inference — to  reflect  the 
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user’s  goals,  and,  vice  vena,  provides  feedlMck  on  diese  goals  through  its  on-going  internal  assessment 
of  supporting  tasks.  It  dynamically  configures  tasks  according  to  the  user's  higfi-level  expository 
purposes  for  die  requested  report  It  dien  moniton  dteir  performance,  and,  if  necessary,  reconfigures 
them  and  provides  feedbadc  to  the  user  on  the  efficacy  of  generating  the  requested  report  In  addition  to 
the  usual  domain  knovWedge,  like  cases  and  heuristic  diagnostic  rules,  FRANK  uses  taxonomies  of 
report  types  and  reasoriirg  strategy  for  generatii^  d^em.  'These  were  also  devdoped  during  this 
rnearch;  they  are  comjd^y  iwveL 

FRANK  produces  various  types  of  reports  (e.g.,  bahmced  pro-con  deddon  brief,  biased  pro-position 
advoca^  brieO  by  using  case-based,  rule-based  and  procedural  reasoning  in  ways  that  reflect  the  user's 
purposes.  For  instance,  if  the  user  specifies  duit  the  infonmation  is  for  a  bakmced  decision  briefing, 
alternative  courses  of  action  are  presented  in  a  neutral  matmer.  In  this  context,  FRANK  would  specify 
that  both  pro  and  con  supporting  cases  aivl  analyses  must  be  retrieved,  atudyzed,  and  incorporated  into 
the  report  FRANK  dynamicalfy  configures  the  supporting  tasks  tweded  to  produce  its  reports,  for 
instance  by  specifying  the  sintilarity  nretric  and  de^tion  of  ‘best"  case  to  be  used  in  the  CBR  nuxlules 
incorporated  into  FRANK.  FRANK  evaluates  the  executed  tasks,  and,  if  necessary,  can  reconfigure  the 
reasoning  mechanisms  and  cause  them  to  try  to  accomplish  th^  tasks  agaitt;  if  all  attempts  fail,  the 
user  is  told  that  the  report  will  be  hard  to  generate  aiul  that  artother  report  type,  selected  from 
FRANK'S  taxonomy,  might  be  more  approfMiate. 

An  obvious  benefit  of  a  system  like  PRANK,  which  combines  different  types  of  reasoning  (case-based, 
rulebased,  etc.)  in  cme  generic  architecture,  is  leverage  and  robustrwss.  'The  different  reasoning 
mecharusms  can  conqdement  and  supplement  each  other.  Another  is  tiie  ability  to  make  supporting 
tasks,  like  case  retrieval,  sensitive  to  the  use  of  the  informatioru  idiat  you  retrieve  and  how  you  reason 
with  it  deperul  on  what  you  are  going  to  do  with  it  FRANK  offers  a  promising  step  in  the  direction  of 
intdl^ent  decision  support  systems  using  conponent  technologies,  such  as  case-ba^  reasoiung  and 
information  retrfoval,  by  making  them  task-sensitive.  For  fostance,  FRANK'S  report  and  report- 
generation  taxonomies  can  be  used  to  guide  infcnmation  retrieval,  where  ktwwledge  of  the  intended  use 
of  information  is  key  to  the  actual  retrieval  of  it. 

Notable  results  of  our  work  on  the  FRANK  Project  are: 

•  dyiuunic  configuration  of  lower  levd,  supporting  tasks  (e.g.,  CBR)  according  to  high-level  tasks 

(e.g.,  report  needs). 

•  internal  monitoring  and  possible  reconfiguration  of  tasks  (e.g.,  CBR)  during  processing. 

•  selection  of  strategies  arid  plans  afiects  the  type  of  CBR  used. 

•  interactive  assistance  to  user  through  suggesticm  of  changes  in  report  type  or  strategy  in  response 

to  previous  cases. 

•  demonstration  of  the  utility  of  a  blackboard  environment  for  integrating  CBR  and  procedural 

knowledge. 

•  identification  of  taxonomies  of  justification  strategies  and  report  types  typical  of  a  diagnostic 

domain. 

(2)  Heuristically  guided  Retrieval  for  Case-Based  Reasoning.  We  explored  various  fundamental  CBR 
issues,  such  as  similarity  assessment  and  case  retrieval,  in  two  ^sterns:  BankXX  and  BROADWAY. 
BROADWAY  is  based  on  a  blackboard  (GBB)  architecture  [Skalak,  1992].  BankXX  is  based  on  the 
classic  architecture  of  heuristic  best-first  search  [Rissland  et  al.,  1^4]. 

The  BANKXX  Project 

BankXX  isa  CBR  system  whose  purpose  is  to  search  out,  harvest,  and  analyze  information  needed  for 
casebased  argument  BankXX  generates  case-based  argument  by  gathering  and  analyzing  various  types 
ot  multiply  irdexed  and  multiply  represented  domain  knowledge.  BankXX  demonstrates  the  use  of 
cla^ic  heuristic  search  as  a  control  regime  for  gathering  and  analyzing  information  for  case-based 
argument  BankXX  is  unique  in  that  it  uses  classic  heuristic  search  with  resource  bounds  to  guide  and 
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constrain  its  reasoning  and  information  gathering  in  a  hi^Uy  connected  directed  graph  of  domain 
informatkm  of  various  types  (e.g.,  actual  caaes,  prototype  cases,  case  citations,  typical  factual 
scenarios,  domain  diemies).  BaidcXX  is  implemented  in  Common  Lisp.  It  is  instantiated  in  the  legal 
domain  ctf  personal.  Chapter  13,  bankrupt^. 

Com|deteiy  novel  about  BankXX  is  its  characterization  of  case-based  argument  as  heuristic  search. 
BanldOCs  process  of  "information-grazing*  in  an  extensive  body  of  domain  knowledge  is  modeled  on  the 
way  a  research  assistant  might  rununage  and  gather  information  in  a  large  library  or  on-line 
information  service  where  information  is  highly  interconnected  through  standard  indexing  schemes, 
case  citation  system^  artd  secondary  sources.  BankXX  is  notaUe  in  its  use  of  multiply  indexed 
information  to  handle  classic  prcfolems,  such  as  too  few  or  too  numy  retrieved  cases,  and  its  at^ty  to 
view  cases  in  light  of  other  knowledge  to  assist  retrieval. 

The  "bottom  up"  process  nnodded  by  BankXX  comfrfements  the  first  research  focus  of  this  grant,  the 
FRANK  Project,  as  wdl  as  our  earlitf  DARPA-sp<msored  projects,  like  CABARET,  which  take  a  "top 
down"  perspective.  Sttce  BankXX  uses  some  of  the  cotranesdal  indexing  methods  (e.g.,  case  citation 
schemes),  it  couki  eventually  be  linked  with  existing  orv-line  data  bases  both  to  extend  its  own 
knowledge  store  and  to  enlumce  the  type  of  information  gathering  fiiey  provide.  Using  CBR  with  IR 
would  help  rennediate  problems  of  both  current  CBR  systems,  such  as  the  knowledge  acquisition 
botfienedc  (e.g.,  small  case  bases),  as  well  as  well-known  shortconungs  of  current  IR  systems,  such  as 
domain-  and  task-igrrorance. 

BankXX  modds  argument  in  terms  of  a  collection  of  a  tfozen  or  so  "argunrent  pieces"  which  fmm  the 
buildfog  Mocks  of  any  precedent-based  argument.  These  indude  supportmg  cases,  best  supporting  cases, 
contrary  cases,  best  ccmtrary  cases,  leading  cases,  factor  analyses,  supporting  dtations,  apfdicable  arvl 
nearly  applicable  legal  theories,  prototype  stories,  etc  [Rissland  et  al.,  1994].  BardcXX  views  argument 
creation  as  the  filling  out  of  thme  argument  pieces.  WMe  this  is  a  very  simplified  representation  of  an 
argunrent,  it  is,  we  fed,  a  prerequisite  to  nnore  sophisticated  representafions,  such  as  those  modeling 
lo^cal  or  rhetorical  structure. 

We  used  three  difierent  evaluation  functions  in  BankXX.  These  encode  knowledge  about  case-based 
argument  at  different  degrees  of  sophistication.  The  evaluation  function  that  is  least  knowledge- 
intensive  uses  only  general  knowledge  about  information  types  (e.g.,  cases,  domain  theories).  The  next 
most  knowledge-rich  evaluation  function  is  based  on  knowledge  of  fiie  argument  pieces  (e.g.,  rxrost  on- 
point  cases  both  pro  and  con,  a  winning  theory,  a  persuasive  {xrototype,  a  stereo^ical  dorruun  story) 
that  go  into  a  go<^  argument.  The  third  evaluation  function  eiKodes  krtowledge  almut  how  to  evaluate 
an  argument  (eg.,  according  to  the  centrality  of  cases  died,  win  record  of  a  chosen  theory  on  similar 
cases). 

During  the  last  funding  period,  we  performed  an  extensive  series  of  empirical  experiments  to  assess 
BankXX's  performance.  These  experiments  fell  into  three  classes:  (1)  evaluation  of  BankXX 
performance  through  comparison  against  domain  expert  performaiKe,  that  is,  by  comparing  BankXX 
output  with  court  opinions;  (2)  evaluation  of  BankXX  as  a  (TBR  program  by  variation  of  several  of 
BankXX's  internal  mechanisms  (e.g.,  evaluation  functions);  and  (3)  comparison  of  BankXX  against 
previous  CBR  programs,  namdy  HYPO,  which  was  also  developed  at  UMass.  This  involved  running 
BankXX  on  already  litigated  cases  and  comparing  its  output  with  the  published  court  opinions.  Thus 
for  our  assessment  is  that  Bank)(X  performs  well  gjven  that  it  is  a  heuristic  bottom  up  approach  to 
argument 

BankXX  is  notable  in  that  it  made  the  following  contributions  to  our  knowledge  about  CBR: 

•  The  process  of  gatholng  information  to  support  an  argument  can  be  modeled  as  heuristic  search. 

•  Case  retrieval  is  effectively  performed  through  a  combination  of  search  and  knowledge-based 

indexing 
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•  The  search  for  cases  and  other  supporting  dtxnaln  knowledge,  such  as  tfieories  or  re-occurring 
scenarios,  can  be  performed  within  a  retrieval  foamewotk. 

The  BROADWAY  Project 

In  additicm  to  BatdcXX,  we  explored  fundamental  CBR  issues  in  the  BROADWAY  project  [Skalak, 

1992].  Ute  BROADWAY  system,  which  was  implemented  in  GBB,  represents  cases  as  knowledge 
sources  in  a  Uadcboard-based  architecture.  It  e)^red  an  alternative  modd  of  case  representation  and 
retrieval.  BROADWAY  was  iirstandated  in  the  donudn  of  BROADWAY  was  instantiated  in  the 
common-sense  domain  of  the  sdection  of  suitaUe  automobile  modds  for  purchase,  given  a  set  of  user 
requiremorts. 

Rather  than  rdying  on  gfobal  metrics  to  assess  case  siirtilarity,  BROADWAY  apjriies  local  similarity 
metrics  that  apfdy  oiUy  in  particular  sub-regions  of  foe  case  space,  for  iirstaiKe,  in  neighborhoods  of 
certain  imteworfoy,  prototypical  cases.  This  approach  is  motivated  by  the  intuitive  idea  that  foe  way 
to  determine  which  cases  are  similar  will  depei^  on  foe  types  arul  particulars  of  cases  being  considered. 
(This  theme  was  also  ex{dored  in  BatfoXX.)  Another  emphasis  of  BROADWAY  was  the  representation 
of  cases  as  dyruonic  knowledge  sources  that  can  recognize  when  they  are  rdevant  to  a  problem  situation. 
Such  an  af^voach  may  be  more  amenaUe  to  paralld  implementations  than  dassic  retrieval  schemes 
wifo  nnore  centralized  OT  uniform  control. 

(3)  Machine  Learning  and  CBR.  We  exfdored  issues  of  nuichine  leanung  and  CBR  in  three  separate 
projects; 

(1)  CABOT 

(2)  learning  prototypical  cases 

(3)  a  study  of  conc^  drift. 

In  foe  CABOT  system  [Callan  et  al.,  1991L  we  explored  foe  use  of  an  inductive  learning  algorifom  to 
adjust  case  retrieval  and  adaptation  mechanisms.  CABOT  was  instantiated  in  the  dassic  ML  domain  of 
Ofodlo.  CABOT  demonstrated  a  50%  reduction  in  foe  mnrtber  of  stored  cases  needed  and  an  increased 
levd  of  game-playing  performance. 


Learning  Prototypical  Cases 

OFF-BROADWAY,  MO  and  RMHC-*  are  three  CBR-ML  tystems  that  leam  case  prototypes.  We  feel 
foat  nnethods  that  enable  foe  learning  of  prototypes  would  represent  a  significant  advatKe  siitce  they 
deaity  can  play  such  a  large  role  in  both  CBR  and  ML.  Sirtce  these  programs  use  a  variety  of  so-call^ 
weak  methods,  such  as  Monte  Carlo  samplii^  random  mutation  hiU-dimbing  and  genetic  algorithms, 
they  are  applicable  to  a  variety  of  doitudns. 

All  three  programs  are  fully  implemented  in  Coiiunon  L^p  and  have  been  tested  on  bendunark 
databases  ftom  foe  University  of  California  at  Irvine  Machine  Learning  Repository,  induding  heart 
disease  arul  breast  cancer  databases. 

OFF-BROADWAY,  MCI  attd  RMHC-*  made  the  following  iwtable  contributions: 

•  Snqrfe  search  techniques  with  a  nmdom  component  can  be  used  to  decrease  substantially  foe  on- 

liM  storage  costs  of  nearest  nei^t^bor  retrieval  algorithms. 

•  Qasdfication  accuracy  can  be  improved  in  some  domains  through  the  identification  of 

prototypes  to  be  used  in  nearest  neighbor  classifiers. 

•  Frature  selection  can  be  performed  using  raitdom  search  techniques  to  further  reduce 

classification  costs. 
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Concept  Drift 

We  have  devdbped  an  approach  using  standard  CBRand  ML  techniques  (e.g.,  ID5R)  to  rerognize 
CMicept  drift,  ttuou^  examination  of  case  examplan  and  the  decision  trees  that  they  can  induce.  In 
pcdiminaiy  work  [Btodley  &  Rissland,  1993L  we  devdoped  a  metric  to  measure  "ii^bilit)^  in 
decision  trees  (or  any  partially  mdered  set).  By  monitoring  dte  stability  of  a  decision  tree  as  it  is 
inocmenlally  generate,  we  can  detect  episodes  of  potent  concept  drift.  These  hy^tl^sized 
episodes  of  drift  can  ttien  be  exannined  mme  fuUy— by  statistical  aiudyses,  by  generation  of  pre-  and 
poat-episode  trees,  etc.— to  determine  if  the  concept  has  indeed  changed.  If  dioe  has  been  drift,  the 
concept  out  be  alteied  aoooidingjiy. 

The  iMlity  to  delect  and  respond  to  concept  drift  is  critical  to  any  tyston  which  deals  with  the  real 
worid.  This  is  e^wdally  true  for  systems  with  some  degree  of  autonomy  (e.g.,  dtose  not  constantly  re- 
ei^juieered  by  human  experts).  Our  approach  of  combining  CBR,  whose  forte  is  knowledge  of  cases,  with 
ML,  yihoae  is  the  induction  of  concept  representations  from  examples  (ie.,  cases),  offers  great  promise  in 
attadUi^  diis  fundamental  problem.  This  researdh  thrust,  devdoped  in  the  last  period  of  this  research 
graid,  is  cunentty  being  purnied  wifo  other  funding  support  and  b  foe  subject  a  new  proposal. 

nCHNOLOGY  TRANSFER:  We  have  disseminated  the  resulb  of  fob  funding  through  a  lai{^  variety 
of  conferences  and  puUications.  For  instance,  resulb  of  fob  researdt  have  been  puUbl^  in  foe 
National  Annual  Conferences  of  the  AAAI,  ^  biannual  international  IJCAI  conferences,  DARPA> 
yonsored  conferences  on  CBR,  Machine  Learning  Conferences,  the  Annual  Cognitive  Science  Conferertce, 
bdemational  Gcmferences  on  AI  and  Law,  and  AAAI-^nsored  workshops  and  tynqposb  cm  CBR,  IR, 
and  ML.  In  addition,  we  have  ported  our  ideas  and  m^ods  to  a  large  NSF-sponsorcri  project  on 
intdUgent  information  retrieval  at  University  of  Massachusetts  at  Amherst.  This  b  a  long  term  (3-6 
year)  project,  on  %tfhich  Rissland  b  a  co-PI,  aind  it  b  sponsored  by  NSPs  Sbte-University-Industry 
Initiative.  Prdiminary  resulb  indicate  that  a  combinki  CBR-IR  apjntMch  can  lead  to  striking  resulb. 

fat  addition,  ideas  from  thb  and  previous  landmark  research,  also  sponsored  through  DARPA,  are  being 
used  by  otiier  researchers  throug^iout  tiie  worid  (e.g.,  several  Australian  AI  and  Law  projeeb).  Ideas 
about  case^nsed  argument  previously  devdoped  tty  our  group  are  iv>  w  being  formalized  in  various  logics 
by  several  AI  researchers  interested  in  abductiem  and  truth-maintenaitce  (e.g.,  Prakken,  Loui,  Gordon). 


SYSTEMS  DEVELOPED  DURING  GRANT  FUNDING 

CABARET  —  the  first  mixed  paradigm  (CBR-RBR)  hybrid.  It  combines  HYFO-style  CBR  with 
standard  (forward  and  backward)  rule-based  reasoning  through  the  use  of  heuristic  control  rules.  The 
control  rules  embody  theory  of  statutory  argument  composed  of  argument  stances,  moves  and  primitives. 
CABAKET-style  argumenb  address  key  issues  in  rule  interpretation,  including  open-textured  predicates 
and  rules  wifo  emei^ent  exceptions  and  unstated  prerequisites. 

•  Rissland,  E.  L.  &  Skalak,  D.  B.  (1991).  CABARET:  Rule  Interpretation  in  a  H3d>rid  Architecture. 
International  Journal  of  Man-MacMne  Studies,  34(6),  839-887. 

•  %alak,  D.  B.  k  Rissland,  E.  L.  (1992).  Argumenb  and  (3ases:  An  Inevitable  Intertwining.  Artificial 
Intelligerux  and  Law:  An  International  Journal,  1(1),  3-48. 

BankXX  —  foe  first  CBR  tystem  using  the  framework  of  heuristic  best-first  search  to  guide 
retrieval  of  cases  and  other  pertinent  knowledge  for  adversarial  argument.  It  searches  through  a  case- 
domain-graph  whose  nodes  represent  key  items  of  legal  knowledge,  such  as  cases,  legal  theories, 
stereotypical  factual  scenarios.  Search  is  guided  by  one  of  three  heuristic  evaluation  functions— called 
node-type,  argument-piece,  argument-dimensions.  Each  bkes  into  consideration  how  the  infoimation  in 
an  (op^  node  can  contribute  to  the  evedving  argunwnt  being  built  up  as  BankXX  searches. 
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•  Rtaland,  E.  L,  Skalak,  D.  B.  k  Friedman,  M.  T.  (1993).  BankXX:  A  Program  to  (Generate  Aigim^t 
through  Case-Baaed  Search.  Prvceeiings,  Fourth  Jntermthnal  Conference  on  Artificial  Intelligence 
and  Lm.  Amsterdam,  The  Netheriands.  ACM. 

•  Riariaiid,  E  L.,  Scalak,  D.  B.  &  Friedman,  M.  T.  (1993).  Case  Retrieval  throu^  Multiple  Indexing 
and  Heuristic  Search.  Proceedings  of  IJCAI-93.  Chambery,  Savoie,  France.  International  Joint 
CUiferences  on  Artificial  Intdligetwe. 

FRANK  —  a  Uadcboard-baaed  architecture  to  create  diagnostic  reports  tailored  to  the  user's 
prescribed  goals  and  specified  report  type.  This  mixed  paradigm  system,  incorporating  CBE  rule- 
based  and  (Arming  components,  dynamically  modified  its  retrieval  strategies  and  queries  urith 
feedbadc  from  die  ^stem's  |xevious  success  or  ^ure  in  retrieving  cases  that  support  a  user's  rhetorical 
and  pragmatic  goals  for  die  repcvt. 

•  Risshmd,  E.  L.,  Daitiels,  J.  J.,  Rubinstein,  Z.  B.  it  Skalak,  D.  B.  (1993).  Case-Based  Diagnostic 
Analysis  in  a  Blackboturd  Architecture.  Proceedings  of  the  Eleventh  National  Conference  on 
Artificial  Intelligence,  66-72.  Washington,  DC.  AAAI  Press/MTT  Press. 

In  additkm  to  diese  CBR  ^sten^  Rissland's  CBR  has  devetoped  a  number  of  CBR-ML  experiments 

and  ^sterns: 

CABOT  —  This  project  in  case-based  search  developed  an  inductive  learning  algorithm  that 
adjusts  its  retrieval  aiul  a^ptation  mechanisms.  CABOT  demonstrated  a  50%  reduction  in  the  number 
of  stored  cases  aiul  an  increased  level  of  performance  in  a  game-playing  aj^lication. 

•  CaHan,  J.  P.,  Fawcett,  T.  E.  it  Rissland,  E.  L.  (1991).  CABOT:  An  Adaj^ve  Approach  to  Case-Based 
Search.  Proceeding,  12th  Internationa!  Joint  Con^rence  on  Artificial  Intelligence,  803-808.  Sydney, 
Australia.  International  Joint  Conferences  on  Artificial  bitelligenoe,  Inc 

BROADWAY  —  This  system  represents  cases  as  blackboard  knowledge  sources  whose 
preccmdltions  invoke  local  similarity  functkms  that  apply  oitly  within  a  closed  neighborhood  of  each 
in  the  space  of  cases.  Broadway  provided  a  new  dynamic  modd  of  case  retrieval  in  which  individual 
cases  are  assigned  their  own  similarity  metrics,  so  tiiat  retrieval  varies  according  to  the  type  of  case 
being  retrieved. 

•  Skalak,  D.  B.  (1992).  Representing  Cases  as  Knowledge  Sources  that  Apply  Local  Similarity  Metrics. 
The  lith  Annual  Conference  of  the  Cognitive  Science  Sodety,  325-330.  Bloomington,  Indiana. 
Lawrence  Eilbaum. 

OFF-BROADWAY  —  A  case  retrieval  and  classification  system  that  learns  a  set  of 
distinguished  cases  (actual  prototypes)  that  have  demoirstrated  classification  power.  The  system 
nnaintains  a  population  of  sets  of  potentially  prototypical  cases;  each  set  is  evaluated  by  its 
classification  accuracy.  A  genetic  algorithm  is  used  to  search  the  space  of  prototype  sets  ("spaiming 
sets*  in  ttte  language  of  §IZ.B)  for  one  of  superior  accuracy. 

•  Skalak,  D.  B.  (1993).  Using  a  CSenetic  Algorithm  to  Learn  Prototypes  for  Case  Retrieval  and 
Oassification.  Proceeding  of  the  AAAI-93  Case-Based  Reasoning  Workshop  (Technical  Report  WS- 
93-01).  Washington,  D.C.  Anwrican  Association  for  Artificial  Intelligence,  Menlo  Park,  CA. 

•  Skalak,  D.B.  (1994). 


ABISS  —  a  blackboard  system  that  integrated  CBR  with  "from-scratch''  first  priiKiples 
problem  sdving  to  predict  integer  sequences.  The  system  was  one  of  tire  first  to  use  a  backboard 
architecture  to  build  a  hybrid  system  that  incorporated  case-based  reasoning  with  other  reasoning 
techniques.  The  program  also  investigated  the  utility  of  case-based  control  strategies  and  investigated 
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the  ap|dioith)n  of  a  number  of  similaiity  metrics  for  case  retrieval  at  different  points  in  the  course 
pioUem  solving. 

•  Riaaland,  E.L.,  Basu,  C,  Danids,  J.,  McCarthy, }.  RuMnstein,  Z.  ic  Scalak,  D.B.  "A  Blackboard-based 
Architecture  for  ^R:  An  Initid  Report*.  Proceedings  of  the  Case-Based  Reasoning  Viorkshop, 
May  1991,  pp.  77-92. 


PUBUCATIONS 

•  Rissland,  E.L.,  Scalak,  D.B.,  and  Friedman,  M.T.  *BankXX:  Suj^rting  Legal  Arguments  through 

Heuristic  Retrieval",  in  preparation  for  submission  to  the  Journal  of  Artificial  Intelligence  and 
Law. 

•  Risslaivl,  EL.,  %alak,  D.B.  and  Friedman,  M.T.  "Evaluating  Legal  Arguments"  in  preparation  for 

submission  to  the  Journal  of  Artificial  Intelligence  and  Law. 

•  9calak,  D.B.  "Prototype  and  Feature  Sdecdm  by  Sampling  and  Random  Mutation  Hill  Climbing 

Algorithms".  To  appear  in  Proceedings  of  the  Eleventh  International  Conference  on  Machine 
Laammg,  New  ftiinswidc,  NJ.  1994.  Morgan  Kaufounn. 

•  RissIaiKi,  E.L.,  Skalak,  D.B.  &  Friedman,  M.T.  "Heuristic  Harvesting  of  Information  for  Case-Based 

Argument"  Proceedings  of  the  Twelfth  National  Conference  for  Artificial  Intelligence  (AAAI-94), 
Seattle,  WA,  (in  press) 

•  Rissland,  E.L.,  Daidds,  J.,  Rubinstein,  Z.,  &  Skalak,  D.B.  "Case-based  Diagnostic  Aiudysis  in  a 

Blackboard  Architecture."  Proceedings  of  the  Eleventh  National  Conference  for  Artificial 
Intettigence  (AAAI-93),  Washington,  D.C.  pp.  66-72. 

•  Rissland,  EL.,  "Legal  Reasoning  and  Artifidal  Intelligence:  Some  recent  advaiKes  with  an  emphasis 

on  case-based  approaches."  Proceedings  First  NISCALE  V/orkshop  on  Criminality  and  Law 
Enforcement.  NettolaiKls  Institute  for  the  Study  of  Qiminality  arid  Law  Enforcement  (NISCALE), 
Dm  Haag,  The  Netherlands,  October  1993,  pp.  187-205. 

•  Rissland,  E.L.,  Danieb,  ).J.,  Rubmstein,  Z.B.  and  Skalak,  D.B.  "Diagnostic  Case  Retrieval  Guided  by 

Evaluation  and  Feedback".  Proceedings  of  the  AAAI-93  Workshop  on  Case-Based  Reasoning, 
Wadiington,  DC,  July  1993,  pp.  135-140. 

•  Rissland,  E.L.,  Scalak,  D.B.  and  Friedman,  M.T.  "U»ng  Heuristic  Search  to  Retrieve  Cases  that 

Support  Arguments.  Proceedings  of  the  AAAI-93  Workshop  on  Case-Based  Reasoning,  Washington, 
DC,  July  1993,  pp.  5-11. 

•  Rissland,  E.L.,  Skalak,  D.B.,  &  Friedman.  M.T.  "BANIOOC  A  Program  to  Generate  Argument 

throu^  Case-Based  Search."  Proceedings  of  the  Fourth  International  Conference  on  AI  and  Law 
(ICAIL-93) ,  Amsterdam,  The  Netherlarkls,  Jime  1993,  pp.  117-124. 

•  Rissland,  EL.,  Skalak,  D.B.,  &  Friedtiuin,  M.T.  "Case  Retrieval  Through  Multiple  Indexing  and 

Heuristic  Search."  Proceedings  of  the  Thirteenth  International  Joint  Conference  on  Artificial 
Intelligence  (IJCAI-93),  Chambery,  Savoie,  France,  August  1993,  pp.  902-908 

•  Bradley,  C.  E.  &  Rissland,  E.  L.  "Measuring  CotKept  Change."  Working  Notes  of  the  AAAI-93  Spring 

Symposium  on  Training  Issues  in  Incremental  Laiming,  1993.  pp.  98-107. 
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•  Skalak,  D.B.  ^Rqucsenting  Cases  as  Knowledge  Sources  that  Apply  Local  Similarity  Metrics''. 

Proceedtrtgs  of  the  Fourteenth  Annual  Conference  of  the  Cogrdtioe  Science  Society,  pp.  325-330. 

1992, 

•  9(alak,  D.B.  and  E.L.  Rissland.,  'TJsii^  Case-Based  Reasoning  to  Extend  the  Expertise  of  Expert 

^sterns*  in  Expert  Systems  in  Law,  A.  Martino  (Ed.),  Elsevier  Science  Publiidiers.  1992.  pp.  321- 
338. 

•  9(alak,  D.B.  it  Rissland,  E.L.  "Arguments  and  Cases;  An  Inevitable  Intertwining."  Artificial 

InidUgence  and  Law  — An  International  Journal,  Kluwor ,  pp.  3-44,  Fall  1992. 

•  Skalak,  D.B.  it  Rissland,  E.L.  "Argument  Moves  in  a  Rule-Guided  Domain."  Proceedings  of  the  Third 

Interrutiondl  Conferenu  on  AI  and  Law  (JCAIL-91),  Oxford,  June  1991,  pp.  1-12. 

•  Callan,  JP.,  Fawcett,  TP.  A  Rissland,  E.L.  "CABOT:  An  Adaptive  Approach  to  Case-Based  Search". 

Proceedings  of  the  Ttoelfth  International  Joint  Conference  on  Artifi^  Intelligence  (IJCAI-91),  pp. 
803-808. 


•  Rissland,  E.  L.,  it  9calak,  D.  B.,  "CABARET:  Statutory  Interpretation  in  a  Hybrid  Architecture." 

International  Journal  of  Man-Machine  Studies  (IJMMS),  June,  1991,  (34):839-887. 

•  RisslaiKl,  E.L.,  Basu,  C.,  Daniels,  J.,  McCarthy,  J.  Rubmstein,  Z.  &  Skalak,  D.B.  "A  Blackboard-based 

Architecture  for  CBR:  An  Initial  Report".  Proceedings  of  the  Case-Based  Reasoning  Workshop, 
May  1991,  pp.  77-92. 


•  Callan,  J.P.,  Fawcett,  T.E.  it  Rissland,  E.L.  "Adaptive  Case-based  Reasoning".  Proceedings  of  the 
Case-Based  Reasoning  Workshop,  May  1991,  pp.  179-190. 


INVITED  LECTURES,  PANELS,  COLLOQUIA  AND  CONFERENCE  PRESENTATIONS 
Rissland 

October  1993  -  Invited  Speaker,  First  NISCALE  Coitference  on  Criminality  and  Law  Enforcement  (The 
Netherlands) 

August  1993  -  International  Joint  Cbnf.  on  Artificial  Intelligence,  fiCAI-93  (Chambery,  France) 

July  1993  -  AAAI-93  (Washington,  DQ 

July  1993  -  AAAI  Woiltshop  on  CBR,  (Washington,  DC) 

June  1993  -  IntematitHial  Conference  on  AI  and  Law,  ICAIL-93  (Amsterdam,  NL) 

May  1993  -  Brartdeis  University 

Mardt  1993  -  AAAI  Sprir^  Symposium  on  CBR  artd  IR  (Stanford) 

Mardi  1993  -  AAAI  ^ring  Symposium  on  Machine  Learning  (Staitiord) 

March  1993  -  Lockheed  Symposium  Series  (Palo  Alto) 

August  1992  —  Series  of  six  lectures  at  Department  of  Electronics,  Research  Labs,  New  Delhi,  India. 
August  1992  —  lEEE/Delhi,  Iixlia. 

July  1992  —  Invited  plenary  lecture  "Recent  Progress  in  AI  and  Legal  Reasoning."  Tenth  National 
CbnferetKe  for  Artificial  Intelligence  (AAAI-92)  San  Jose,  CA. 

May  1992  —  Daylot^  tutorial  on  CBR,  Hartford  Graduate  Center. 

May  1992  -  CBR  Roundtable,  Boston,  MA 

November  1991  -  Distinguished  Lecture,  University  of  Wisconsin  (2  talks) 

June  1991  -  International  Cortferenoe  on  ^  arxl  Law,  ICAIL-91  (Oxford  England) 

May  1991  -  DARPA  Case-based  Reasoning  Workshop  (Washington,  EXT) 
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Mardi  1991  •  AAAl^vingSynqposium  ( Pak>  Alto,  CA) 

FelMniaiy  1991  -  Apf^  Cnpmte  Research  Labs  (Cupertino,  CA) 

Skalak 

July  1992  -  Fourteenth  Annual  Conference  of  the  Cognitive  Science  Society,  Bloomington,  IN 
RuUnstein 

July  1993  -  AAAI-93  -  Case-Based  Diagnostic  Analysis  in  a  Blackboard  Architecture.  -  Washington, 
D.C. 

Friedman 

July  1993  -  AAAI  Workshop  on  CBR  -  Waritington  DC 
Daniels 

July  1993  -  AAAI  CBR  Workshop  -  Diagirostic  Case  Retrieval  Guided  by  Evaluation  arul  Feedback 


Omfeccnces  Attended: 

Rissland 

January  1994  -  CBR  Roundtable,  Cambridge  MA 

August  1993  -  International  Joint  Cbnf.  on  Artificial  Intelligence,  IJCAI-93  (Chambery,  France) 

July  1993  -  AAAI-93  (Washington,  DO 

Juty  1993  -  AAAI  Workriiop  on  CBR,  2  talks  (Washington,  DC) 

June  1993  -  IntOTiatumal  Conference  on  AI  and  Law,  ICAIL-93  (Amsterdam,  NL) 

Mardr  1993  -  AAAI  Sprii^  Symposium  on  CBR  and  IR  (Stanford) 

Mardr  1993  -  AAAI  ^ring  Symposium  on  Machine  Leamii^  (Stanford) 

Mardi  1993  -  CBR  Roundtable,  Boston,  MA 

July  1992  -  Invited  plenary  lecture  ^Recent  Progress  in  AI  and  Legal  ReasoiUng." 

Tentii  Natirmal  Conference  for  Artifidal  Intelligence  (AAAI-92)  San  Jose,  CA 
May  1992  -  CBR  Roundtable,  Boston,  MA 

June  1991  -  International  Conference  On  AI  and  Law,  ICAIL-91  (Oxford  England) 

May  1991  -  DARPA  Case-based  Reasoning  Workshop  (Washington,  DQ 
March  1991  -  AAAI  Sjning  Symporium  ( Palo  Alto,  CZA) 

October  1991-  CBR  Roundtable,  Amherst,  MA 

Skalak 

January  1994  -  CBR  Roundtable,  Canibridge  MA 

July  1993  -  Eleventh  National  inference  on  Artifidal  Intelligence,  Washington,  DC 
July  1993  -  AAAI  Case-Based  Reasoning  Workshop,  Wadiington,  DC 

June  1993  -  Fourth  Intornational  (Conference  on  Artifidal  Intelligence  and  Law,  Amsterdam,  The 
Netherlands 

April  1993  -  The  Second  International  Conference  on  Artifidal  Intelligence  Applications  on  Wall 
Street,  New  York,  NY 

July  1992  -  Fourteenth  Annual  Conference  of  the  Cognitive  Sdence  Sodety,  Bloomington,  IN 

March  1993  -  Toith  International  Conference  on  Machine  Learning,  Amherst,  MA 

June  1991  -  Third  Intematirmal  Conference  on  Artifidal  Intelligence  and  Law,  Oxford,  England 
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May  1991  •  Third  DARPA  Case-Based  Reasoning  Woiicshop,  Washington,  IX 
March  1993  -  CBRRoundtaMe,  Boston,  MA 
Mi^  1992  -  CBR  RoundtaUe,  Boston,  MA 
October  1991  <CBR  Roundtal^,  Amherst  MA 

Riddnilein 

July  1993  -  AAAI  CBR  Workshop  -  Washington  DC. 

July  1993  -  AAAI-93  (Washington,  DO 

May  1991  -  DARPA  Case4)ased  Reasoning  Workshop  -  Washington,  IX 
Friedman: 

January  1994  -  CBR  Roundtable,  Cambridge  MA 

June  1993  -  International  Conference  on  AI  and  Law,  ICAIL-93  (Amsterdam,  NL) 
July  1993  -  AAAI  CBR  Workshop  -  Washington  DC 
Miundil993  -  CBR  Roundtable,  Boston,  MA 
May  1992  -  CBR  Roundtable,  Bmton, 

October  1991  •  CBR  Roundtable,  Arr^erst  MA 

June  1991  -  fetemational  Ccmfetimoe  on  AI  aiKi  Law,  ICAIL-91  (Oxford  England) 
May  1991  -  DARPA  Case-based  Reasoiung  Workup  -  Washington,  DC 

Daniels: 

July  1993  -  AAAI  CBR  Workshop  -  Washington  DC 

May  1991  -  DARPA  Case-based  Reasoiung  Workshop  -  Washington,  DC 
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